Expectation-MiniMax: A General Penalized Competitive Learning Approach to Clustering Analysis

In the literature, the Rival Penalized Competitive Learning (RPCL) algorithm (Xu et al. 1993) and its variants perform clustering analysis well without knowing the cluster number. However, such a penalization scheme is heuristically proposed without any theoretical guidance. In this paper, we propose a general penalized competitive learning approach named Expectation-MiniMax (EMM) Learning that describes a Kullback-Leibler divergence contrast function by an approximate one with a designable error term. Through adaptively minimizing a specific error term meanwhile maximizing the approximate contrast function, we show that the EMM automatically penalizes all rivals during the adaptive competitive learning. Actually, such a rival penalization learning is an alternative way to optimize the contrast function of the Expectation-Maximization (EM) algorithm with at least two advantages: (1) faster speed of model parameter learning, and (2) automatic model selection capability. We present the general learning procedure of the EMM, and demonstrate its outstanding performance in comparison with the EM under a finite Gaussian mixture model. Furthermore, a fast implementation of EMM is presented, which not only includes the RPCL and its Type A variant, but a new variant is also proposed. Particularly, a novel mechanism to dynamically control the rival-penalized forces is given out. We therefore propose a stochastic rival penalized competitive learning, which circumvents the selection problem of the so-called delearning rate in the RPCL and its variants. The experiments have successfully shown its outstanding performance.